Recently, Chain-of-Thought (CoT) prompting has delivered success on complex
reasoning tasks, which aims at designing a simple prompt like ``Let's think
step by step'' or multiple in-context exemplars with well-designed rationales
to elicit Large Language Models (LLMs) to generate intermediate reasoning
steps. However, the generated rationales often come with mistakes, making
unfactual and unfaithful reasoning chains. To mitigate this brittleness, we
propose a novel Chain-of-Knowledge (CoK) prompting, where we aim at eliciting
LLMs to generate explicit pieces of knowledge evidence in the form of structure
triple. This is inspired by our human behaviors, i.e., we can draw a mind map
or knowledge map as the reasoning evidence in the brain before answering a
complex question. Benefiting from CoK, we additionally introduce a
F^2-Verification method to estimate the reliability of the reasoning chains in
terms of factuality and faithfulness. For the unreliable response, the wrong
evidence can be indicated to prompt the LLM to rethink. Extensive experiments
demonstrate that our method can further improve the performance of commonsense,
factual, symbolic, and arithmetic reasoning tasks.

引入了一种新的 Chain-of-Knowledge (CoK) 提示方法，旨在引导强大的语言模型（LLMs）生成显式的知识证明作为结构三元组，从而改进 Commonsense、事实、符号和算术推理任务的性能，并引入 F^2-Verification 方法估计链的可靠性